{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Using variational inference to estimate the existence of objects and their redshifts from COSMOS2020\n", "\n", "Tom Charnock\n", "\n", "Institut d'Astrophysique de Paris\n", "

\n", "\n", "Slides available at presentations.charnock.fr/ppzeds
\n", "

\n", "
\n", "
\n", " \"Sorbonne\n", "
\n", "
\n", " \"Alliance\n", "          Émergence\n", "
\n", "
\n", " \"IAP\"\n", "
\n", "
\n", " \"CNRS\"\n", "
\n", "
\n", " \"Aquila\"\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## AIMS\n", "\n", "##### Target rare objects in deep fields, such as COSMOS2020\n", "##### Accelerate science using machine learning by obtaining new information\n", "##### Avoid scientific bias by proposing only a searching algorithm and not \"results\"" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "
\n", " \"Targetted\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## How do we choose where to target follow-up observations?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "##### Build a probabilistic model of known spectroscopic objects\n", "##### Condition the model on pixel values from photometric surveys\n", "\n", "##### This is a data model and not a physical model... any physics is correlation and not causation" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Catalogue of known objects from spectroscopy\n", "\n", "
\n", " \"Spectroscopic\n", "
\n", "
\n", " $$i = \\textrm{RA}\\text{ and }\\textrm{Dec}$$\n", "

RA and Dec of $\\sim$ 20,000 spectroscopically observed objects, $\\textsf{obj}_{\\textsf{spec},i}$

\n", " $$\\begin{align*}\n", " \\textsf{obj}_{\\textsf{spec},i}&\\sim P(\\textsf{obj}|\\textsf{spec}_i)\\\\\n", " &=\\delta(i)\\,\\,\\,\\,\\forall i\\in\\textrm{Catalogue}\n", " \\end{align*}$$\n", "

Error model for redshifts, $z_i$, given the spectroscopy, $\\textsf{spec}_i$

\n", " $$\\begin{align*}\n", " z_{\\textsf{spec}, i} &\\sim P(z|\\textsf{spec}_i)\\\\\n", " &\\approx\\mathcal{N}(\\mu_{z_{\\textsf{spec},i}}, \\sigma_{z_{\\textsf{spec},i}})\n", " \\end{align*}$$\n", "
\n", "
\n", "

Joint distribution - $\\displaystyle\\textsf{obs}_{\\textsf{spec},i}, z_{\\textsf{spec}, i} \\sim P(\\textsf{obj}, z|\\textsf{spec}_i)$

\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Modeling the existance of objects and their properties\n", "\n", "### We could model the probability of redshifts of objects from pixel values in photometric surveys, $\\textsf{pix}_i$\n", "\n", "$$z_{\\textsf{phot},i}\\sim P_\\mathscr{w}(z|\\textsf{obj}_i,\\textsf{pix}_i)$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "### In fact we can model the joint distributions\n", "\n", "$$z_{\\textsf{phot},i}, \\textsf{obj}_i \\sim P_\\mathscr{w}(z, \\textsf{obj}|\\textsf{pix}_i)$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "### Which comes from the conditional distribution using the chain rule\n", "$$P_\\mathscr{w}(z|\\textsf{obj},\\textsf{pix}_i) = P_\\mathscr{w}(z|\\textsf{obj}_i, \\textsf{pix}_i)P_\\mathscr{w}(\\textsf{obj}|\\textsf{pix}_i)$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "### Or the marginals\n", "$$P_\\mathscr{w}(z|\\textsf{pix}_i) = \\int_\\textsf{OBJ} \\hskip-0.6em d\\textsf{obj}\\,P_\\mathscr{w}(z| \\textsf{obj}, \\textsf{pix}_i)P_\\mathscr{w}(\\textsf{obj}|\\textsf{pix}_i)\\,\\,\\,\\,\\text{and}\\,\\,\\,\\,P_\\mathscr{w}(\\textsf{obj}|\\textsf{pix}_i)=\\int_\\textsf{REDSHIFT} \\hskip-2.5em dz \\,P_\\mathscr{w}(\\textsf{obj}|z, \\textsf{pix}_i)P_\\mathscr{w}(z|\\textsf{pix}_i)$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Targetting rare objects\n", "\n", "##### Calculate the marginal distribution for the probability of objects given particular properties, i.e.\n", "\n", "### High redshift objects, i.e. $z > 6$\n", "\n", "$$P_\\mathscr{w}(\\textsf{obj}|\\textsf{pix}_i, z>6)=\\int_6^\\infty dz \\,P_\\mathscr{w}(\\textsf{obj}|z, \\textsf{pix}_i)P_\\mathscr{w}(z|\\textsf{pix}_i)$$\n", "\n", "### Naïvely seach for maximal probability of objects\n", "\n", "$$\\hat{i} = \\underset{i\\in\\textrm{Survey}}{\\textrm{argmax}}~P_\\mathscr{w}(\\textsf{obj}|\\textsf{pix}_i, z>6)$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "### This could be effectively extended to any desired attributes" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# The data model" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Select patches/pixels from COSMOS2020 by RA and Dec\n", "\n", "
\n", "

HSC $g$, $r$, $i$, $z$, $y$

\n", "

ULTRAVista $y$, $h$, $j$, $ks$

\n", "

CFHT u* 1, 2

\n", "

Error model for pixel values, $\\textsf{pix}$:

\n", "
\n", " $$\\begin{align*}\n", " \\textsf{pix}_i &\\sim P(\\textsf{pix}|\\{\\text{CCD read out}\\}_i)\\\\\n", " & \\approx \\mathcal{N}(\\boldsymbol{\\mu}_{\\textsf{pix},i},\\boldsymbol{\\Sigma}_{\\textsf{pix},i})\n", " \\end{align*}$$\n", "
\n", "

Could use patches or single pixels


\n", "

We've found little difference between the two forms of inputs

\n", "
\n", "
\n", " \"COSMOS2020\n", " \"COSMOS2020 \n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Assume a model of the objects given the input pixels at the same position as objects in the spectroscopic catalogue\n", "\n", "$$\\begin{align*}\n", "\\textsf{obj}_i&\\sim P(\\textsf{obj}|\\textsf{pix}_i)\\\\\n", "&=\\delta(i)\\,\\,\\,\\,\\forall i\\in\\textrm{Catalogue}\n", "\\end{align*}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "

Build model of suspected background

\n", "
\n", " $$\\begin{align*}\n", " \\textsf{pix}_{\\textsf{background},i}&\\sim \\mathcal{N}(\\boldsymbol{\\mu}_{\\textsf{background}}, \\boldsymbol{\\Sigma}_{\\textsf{background}})\\\\\n", " \\textsf{obj}_{\\textsf{background},i}&\\sim P(\\textsf{obj}|\\textsf{pix}_{\\textsf{background},i})\\\\\n", " &=0\n", " \\end{align*}$$\n", "
\n", "
\n", " $$\\begin{align*}\n", " z_{\\textsf{background},i}&\\sim \\textrm{Uniform}(0, \\infty)\\\\\n", " z_i&\\sim P(z|\\textsf{obj}_{\\textsf{background},i}=0, \\textsf{pix}_{\\textsf{background},i})\\\\\n", " &=1/\\infty\n", " \\end{align*}$$\n", "
\n", "
\n", " \"HSC\n", "
\n", "
\n", " \"HSC\n", "
\n", "
\n", " \"HSC\n", "
\n", "
\n", " \"HSC\n", "
\n", "
\n", " \"HSC\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# The model" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Model as a simple single parameter distribution, such as Bernoulli\n", "\n", "$$P_\\mathscr{w}(\\textsf{obj}|\\textsf{pix}_i) = \\textrm{Bernoulli}(p(\\textsf{pix}_i, \\mathscr{w}))$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Modelling the probability of redshifts given the existance of objects\n", "\n", "##### Flexible enough to model the fact that the distribution of objects is skewed\n", "\n", "##### Simple enough that we can perform analytical integrals\n", "\n", "### Mixture of Gaussians\n", "\n", "$$P_\\mathscr{w}(z|\\textsf{obj}_i, \\textsf{pix}_i) = \\sum_k \\alpha_k(\\textsf{pix}_i, \\mathscr{w})\\hskip0.1em\\mathcal{N}\\hskip-0.25em\\left(\\mu_k(\\textsf{pix}_i, \\mathscr{w}), \\sigma_k(\\textsf{pix}_i, \\mathscr{w})\\right)$$ " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Distribution of spectroscopic redshifts in a spectroscopic catalogue\n", "\n", "##### According to data, there should always be more likely to have objects with redshifts between 0 and 1\n", "\n", "
\n", " \"Spectroscopic\n", "
\n", "

The first fits the data better, but we can't see it because the mean and mode are not good measures...

\n", "

If we want a flat prior over physical properties, we need to reweight the effect of the data

\n", "
\n", "
\n", "\n", "
\n", "

The mean of the data distribution
(for a single Gaussian)

\n", "
\n", " \"Unweighted\n", "
\n", "
\n", "\n", "
\n", "

The mean of the (approximate) physical distribution
(for a single Gaussian)

\n", "
\n", " \"Weighted\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "
\n", " \"Redshift\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Minimise the KL divergence between the joint distribution of the existance of objects and their redshifts from spectroscopy\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "$$\\begin{align*}\n", "\\textrm{KL}(P||P_\\mathscr{w}) & = \\left\\langle\\ln\\frac{P}{P_\\mathscr{w}}\\right\\rangle\\\\\n", "&=\\sum_{i\\in\\textrm{Catalogue}} \\hskip-0.5em\\lambda_i\\big[\\ln P(z=z_{\\textsf{spec},i}|\\textsf{spec}_i) - \\ln P_\\mathscr{w}(\\textsf{obj}=1, z=z_{\\textsf{spec},i}|\\textsf{pix}_i)\\big]\\\\\n", "&\\phantom{=}+\\sum_{i\\in\\textrm{dim}[\\textrm{Catalogue}]} \\hskip-1.2em\\big[\\ln P(z=z_{\\textsf{background},i})- \\ln P_\\mathscr{w}(\\textsf{obj}=0, z=z_{\\textsf{background},i}|\\textsf{pix}_{\\textsf{background},i})\\big]\n", "\\end{align*}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "##### Training\n", "\n", "1. Sample\n", " 1. $\\{z_{\\textsf{spec}, i\\color{gray}{, j}}\\sim\\mathcal{N}(\\mu_{z_{\\textsf{spec}, i}}, \\sigma_{z_{\\textsf{spec}, i}})~|~i\\in\\textrm{Catalogue}\\color{gray}{,~j\\in[1, \\textrm{MC samples}]}\\}$\n", " 2. $\\{\\textsf{pix}_{i}\\sim\\mathcal{N}(\\boldsymbol{\\mu}_{\\textsf{pix},i}, \\boldsymbol{\\Sigma}_{\\textsf{pix}, i})~|~i\\in\\textrm{Catalogue}\\}$\n", " 3. $\\{z_{\\textsf{background}, i\\color{gray}{, j}}\\sim\\textrm{Uniform}(0,\\infty)~|~i\\in\\textrm{dim}[\\textrm{Catalogue}]\\color{gray}{,~j\\in[1,\\textrm{MC samples}]}\\}$\n", " 4. $\\{\\textsf{pix}_{\\textsf{background}, i}\\sim\\mathcal{N}(\\boldsymbol{\\mu}_{\\textsf{background}}, \\boldsymbol{\\Sigma}_{\\textsf{background}})~|~i\\in\\textrm{dim}[\\textrm{Catalogue}]\\}$\n", "2. Calculate\n", " 1. $\\{\\lambda_i\\ln\\mathcal{N}(z=z_{\\textsf{spec}, i\\color{gray}{, j}}|\\mu_{z_{\\textsf{spec},i}}, \\sigma_{z_{\\textsf{spec},i}})~|~i\\in\\textrm{Catalogue}\\color{gray}{,~j\\in[1,\\textrm{MC samples}]}\\}$\n", " 2. $\\{\\lambda_i\\ln P_\\mathscr{w}(\\textsf{obj}=1,z=z_{\\textsf{spec}, i\\color{gray}{, j}}|\\textsf{pix}_i)~|~i\\in\\textrm{Catalogue}\\color{gray}{~,j\\in[1,\\textrm{MC samples}]}\\}$\n", " 3. $\\{\\ln\\textrm{Uniform}(z=z_{\\textsf{background}, i\\color{gray}{, j}}|0,\\infty)~|~i\\in\\textrm{dim}[\\textrm{Catalogue}]\\color{gray}{,~j\\in[1,\\textrm{MC samples}]}\\}$\n", " 4. $\\{\\ln P_\\mathscr{w}(\\textsf{obj}=0,z=z_{\\textsf{background}, i\\color{gray}{, j}}|\\textsf{pix}_{\\textsf{background},i})~|~i\\in\\textrm{dim}[\\textrm{Catalogue}]\\color{gray}{,~j\\in[1,\\textrm{MC samples}]}\\}$\n", "\n", " $$\\textrm{KL}=\\sum_{i\\color{gray}{, j}}(A - B) + \\sum_{i\\color{gray}{, j}}(C - D)$$\n", "3. Minimise the KL divergence\n", " $$\\widehat{\\mathscr{w}} = \\underset{\\mathscr{w}\\in\\mathscr{W}}{\\textrm{argmin}}~\\textrm{KL}$$\n", " " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "

Targetting objects by maximising the probability of particular properties over entire field

\n", "\n", "$$\\hat{i} = \\underset{i\\in\\textrm{Survey}}{\\textrm{argmax}}~P_\\mathscr{w}(\\textsf{obj}, z|\\textsf{pix}_i)$$\n", "\n", "
\n", " \"Maximising\n", "
\n", "\n", "

A way to find rare objects

" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Accelerates science but doesn't bias it.\n", "\n", "# It is essential that this is used *only* for targetting or forcasting\n", "\n", "#### There is no way to assign *a priori* or *a posteriori* belief to any neural model,
therefore they are not good predictive models and so the results can't be used as scientific products!" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Future steps\n", "\n", "##### Larger spectroscopic catalogues with better error models\n", "##### More photometric bands for more information to distinguish the model\n", "##### More properties, such as galaxy or other object type to help in the hunt for rare objects in deep fields" ] } ], "metadata": { "celltoolbar": "Diaporama", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.9" }, "rise": { "controls": false, "height": "100%", "scroll": true, "width": "100%" } }, "nbformat": 4, "nbformat_minor": 4 }